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ABSTRACT 

This paper deals with two problems confronting the 
applied researcher using a behavior analysis approach. The first is 
concerned with the need for independence of behaviors, subjects and 
settings. The effectiveness with which the researcher can solve this 
type of problem depends on the researcher's emphasis on the_ 
examination of procedures, and not experimental design. Similarly, 
the external validity problems involved in reactive experimental 
arrangements and multiple treatment interference are experimental 
procedure problems rather than design problems. Confronting those 
problems in this manner enables the applied researcher to add to the 
bank of empirical knowledge on human behavior. An extensive appendix 
which reviews the sources of invalidity proposed by Campbell and 
Stanley suggests that many are inappropriate to behavior analysis 
designs. Seven designs are presented graphically. (HG) 
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Strong Designs for Behavior Analysis 



O 

Michael Medland, Walter G. Hapkiewicz, and John Molidor 
Michigan State University 

CO 

The use of multiple baseline designs in evaluating behavior analysis research 



in applied settings has been criticized for a number of reasons. This paper represents 
an attempt to examine two of the major criticisms and pose a clarification. The 
first criticism has been called the need for independence of behaviors, subjects, 
and settings. The second concerns the problem of external validity. It is the 
contention of the authors that these two problems will continue to disturb applied 
researchers until they are aware of their relation to basic research and until they 
see the correspondence between experimental de&igu and experimental procedures, 

THE BEHAVIOR INDEPENDENCE PROBLEM <- 
The behavior independence problem has been clearly stated by Kazdin (1973) as 
follows : 

A major area of concern in using this (multiple Baseline) 
gliS^o des ign is that one must be reasonably assured beforehand that the 

a a O 9 a UJ ft— 

f UJ 5 2 Ui ^ 

5So ^Szo'^y target behaviors used are not interdependent or interrelated with 

UJ<iii UJ^5a^^^ 
iitK ujuj^OQtfa 

"ai^^jS each other. In such a situation, inq)lementing a contingency for 

*H<p z^a^^^^ the performance of one behavior may be expected to alter the 



°2i ^^^tm behavior(s) for which continued baseline da^r^ are collected. 



O n UJ ~ O / 

lI^^^^S (p. 519, emphasis ours) 



The problem is said to be applicable not only to Lv^haviors but to individuals 
(one individual being observed may influence another observed individual) and sit- 
CO uations (behavior changed in one situation may influence that 'behavior in some other 

^ place.) It is further contended that because of the possible changes in the other 

observed behaviors, situations, and/or subjects, the miultiple baseline designs lose 
the power to demonstrate the effects of the contingencies. This is said to result 
in an experimental design problem. 
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Kazdin's only recommendation was a return to control group research methods 
on occasions where correlated behaviors, etc. were a problem. But as Sidman (1960, 
1973) has pointed out this is helpful only at the most primative level of analysis . 
when behavioral control techniques are not available. 

The solution to the problem can be found in two perspectives Involving the 
applied researchers view of where his or her research fits into the complex of 
empirical knowledge regarding human behavior. The first perspective involves the 
distinction between experimental design and experimental procedures, and the second 
the age old dichotomy between basic and applied research. 

Designs and Procedures Perspective 

The major point in the distinction between designs and procedures is that if 
one views the "design" as the problem confounding interpretation of results, then 
one fails to examine procedure. 

The multiple baseline design is a time-series design, and the researcher has 
considered it empirically important to represent behavior change in more than two 
dependent variables in order to demonstrate a cause and effect relationship, and/or 
follow the change in behaviors as related to the change in independent variables. 
The same rationale can be applied to any of the other time series designs. 

The power of the behavior analysis approach is in its continuous recording of 

behavior over time and the specification of behavioral control techniques (procedures). 

It is the application of control techniques over time that allows for the demonstration 

of a functional relationship between behavior and the manipulated variables. Thus, 

procedures are considered the first aspects of research, the portion of the research 

that specifies the variables, both independent and dependent, that will be examined 

and the nature of the relationship they will have to each other during different 

experimental phases. The result of taking this procedural perspective is to teduce 

design to a method of representing behavior over time as related to procedural 

modifications. The lack of this perspective results in a loss of problem solving 

behavior on the part of the investigator. The researcher's forgotten strategy is 

O 

O the examination of behavior control procedures. ^ 
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In order to establish and maintain the procedural perspective, it is necessary 
to differentiate between micro and macro research procedures. Micro procedures 
involve the quantitative, qualitative, temporal, and/or special manipulation of 
independent variables within an experimental phase. For example, one may vary 
reinforcement to shape a particular psychomotor skill. The behavioral control 
technique used is differential reinforcement which involves a temporal manipulation 
of the independent variable that is reciprocally dependent on the change in the 
behavior under investigation. 

Macro procedures involve the elimination or presentation of behavioral control 
techniques. For example, after the above psychomotor skill was established and 
stable, the investigator may be interested in seeing how it extinquishes. He does 
this by the elimination of one set of control techniques and the establishment of 
another. This is typically called an experimental phase change, but the important 
point to remember is that it is a procedural manipulation. 

The distinciton between micro and macro procedures is not free from some possible 
ambiguity. But most of that ambiguity can be eliminated by a review of a researcher's 
questions of interest as related to the procedures used. With or without ambiguity 
the distinction puts the emphasis were it belongs: on procedures. 

Keeping the above design ~ procedure perspective in mind, it ii possible to 
clarify the interdependence problem as posed by Kazdin. If the researchers are 
interested in eliminating the interdependence of subjects, they can do so simply by 
incorporating procedures that would at least physically separate the subjects. But 
if they are interested In the interactive process and wished to determine the 
controlling variables, they would incorporate different procedures. The difference 
between the two pieces of research is not a difference in experimental design, but 
a difference in experimental procedure which is governed by the research question asked. 

An example involving interrelated behaviors and settings could be one where 
the iti' archer is interested in the process- of generalization of a skill, like the 
learning of a problem solving- strategy . But another researcher could be for some 



reason interested in keeping skill and situational transfer to a minimum. Both 
researchers will develop procedures to solve and/or examine their specific questions 
but for each the multiple baseline design would be appropriate. It would be 
appropriate for the first because he or she mould follow the course of change for the 
multiple behaviors in multiple settings. For the second it would allow for the 
assessment of the procedures that were designed to reduce interrelatedness . In 
either case a failure (however it is defined) would lead to a change in procedure 
not design. 

Essentially, what one is doing when one complains about correlated or inter- 
related beiiaviors, subjects and settings, is blaming the behaviors of interest for 
not doing what the researcher wanted them to do. Now, if behavior is lawful, it seems 
strange that the researcher should blame the experimental design. In most areas of 
science, the researcher would follow the course of events and see what occurs, (i.e., 
to hold or/change procedures depending on how the data change the researcher's 
experimental questions). Here the problem becomes one of investigating the 
controlling variables by the manipulation of variables thought to be in control 
of the observed changes in behavior. The encounter with such a phenomenon could lead 
to possible serendipitous discoveries and Skinner's (1974, p. 195) advice becomes 
meaningful; drop everything and follow the data. 

Basic and Applied Perspective 

Skinner's advice brings us to the relationship between basic and applied 
research, a relationship we feel is in need of a change in perspective if the applied 
researcher is to deal effectively with problems like those posed by Kazdin. The 
change in perspective deals with whaf. the applied researcher has to add to the complex 
of human knowledge. Everyone agrees that it is the applied researcher's job to deal 
with socially relevant problems with knowledge gained from basic research. With only 
a quick glance one can see a multitude of social problems which involve human behavior 
but another glance will tell you that not a great deal of basic knowledge about human 
behavior is known. The correlated behavior problem is a clear example. Here the 
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applied researcher is dealing with a complex phenomenon that is not understood in 
terms of its controlling variables. 

At present, attempts are being made to deal in an applied way with phenomenon 
that need to be treated in terms of their controlling variables- Such related 
questions can be considered primarily basic research questions only because the 
applied researcher cannot drop everything- But what is it that the applied researcher 
can contribute and how does applied research relate to basic research? 

The answer to the what and the how are closely related- As Skinner (1972) has 
pointed out the practical implications of applied research has lead the applied 
research to deal with variables the basic researcher would have avoided until a later 
date when more related knowledge would be available- For example, the improvement in 
the life of retardates and psyfiotics has made it necessary to deal with more variables 
than convenience dictates and is feasible in the laboratory- The result is that 
discoveries are made that laboratory practices would have made impossible, but which 
will at a future date facilitate laboratory practices in dealing with more complex 
phenomenon. Thus, the relationship is reciprocal. The applied researcher uses basic 
research findings to help develop procedures that are applied in more complex 
situations which can yield discoveries that can be used later by the basic researcher 
in his scientific endeavors. 

Not only is there a possibility for the applied researcher to add to the 
knowledge of the field but in behavior analysis research, it is facilitated by the 
fact that the basic research model is the same as the applied treatment model; the 
basic research manipulation is the applied treatment manipulation, and the basic 
evaluation procedure is the applied evaluation procedure. The differences in the 
two are primarily in the details of the questions to be asked and the related 
specificity of the experimental procedures. This is a well-known fact, but one 
that appears to be over looked by researchers. If it were not overlooked, we would 
not have a concern for avoiding phenomena like correlated behaviors in applied behavior 
analysis research. g 



THE EXIERMAL VALIDITY PROBLEM 
Designs and Procedure Perspective 

The second complaint, also mentioned by Kazdin (1973), is the problem of external 
validity and the restrictions it places on the ability to generalize to other behaviors, 
subjects, and settings. Two possible confoundings are said to restrict the ability 
to generalize the findings. The first is multiple treatment interference. This refers 
to the possibility that a conclusion derived from a later treatment is dependent upon 
previous phases because the effects of each are not erasable. The second, called the 
reactive effects of experimental arrangements, refers to the fact that the particular 
experimental arrangements may preclude generalization to treatment effects across 
time, situations, and subjects. For example, the use of observers in the classroom 
and how their presence may change the behavior of the student irrespective of the . 
treatment. 

Here again, a reactive arrangement is not an experimental design problem but an 
experimental procedure problem. The concern is not for altering the multiple base- 
line design or what ever time-series design is used, but for modification of the 
experimental procedures. The process of observing by using observers could be 
eliminated not by changing the experimental design but by using different instrumentation 
procedures or by collecting data that shows that the observers are not affecting 
events. Following the former procedure often requires waiting for technological 
advances. Using the latter procedure may add confirmation to previous findings 
regarding the use of observers* 

The multiple treatment interference concern is perhaps the kind of problem 
where the applied researcher can offer the greatest contribution to basic knowledge. 
This is because the phenomenon is a historical problem for which time-series designs 
are perfectly suited. The applied researcher^s aim, for example, would be to set up 
a complex educational program and take "detailed" readings of the evolution of various 
phases and behaviors. The researcher would evaluate and adjust the various coiiq)onents 
of the phases until the terminal phase yielded a range of outcomes that meets social 



needs, so in the end the domain of inputs would be related to a range of outputs. 
The functional relation may not be spelled out but the major components will have 
been specified so the basic researcher will not have to grope in the dark. The 
practical applications in real life have given a general description of change over 
time, a general set of potentially relevant variables, and perhaps some measurement 
instruments. 

The outcome of eliminating the above and other possible confoundings leads 
to clear procedural interpretations which represent facts about the world. If 
these facts can be related to other facts, then functional relations can be established 
for subjects, variables, processes and/or the methodologies involved. Thus, for the 
behavior analysis researcher, the problem is not one of external validity but one of 
the clarity of procedure. If no confoundings can be detected, the research results 
either add to or restrict the generality of one or more functional relationships. 

Basic and Applied Perspective 

This brings us to two points that should fully clarify the relation between 
basic and applied research. The first is to say that if one looks to the other 
sciences with a technology it becomes evident that there is no basic-applied dichotomy. 
At all levels from the most theoretical to the most technological there is a trading 
of findings. Perhaps the clearest example of this is in the physical sciences where 
the instruments developed by technology allow for the confirmation of some abstract 
theory. In the behavioral sciences the trade off has even greater potential because 
technological techniques applicable to designing environments could far outstrip 
behavior theory. 

The second and most important point is the choice applied behavior analysis 
researchers must make with regards to their future activities. Basically there are 
two alternatives. The first, is the continued demonstration of the applicability of 
techniques and processes discovered in basic research to human activities. The 
emphasis has been on short-term research With a rigorous demonstration of cause and 
effect. The second alternative is to deal with clearly stated social problems in 
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which research procedures are used to evaluate the outcomes in the sense of adjusting 
the variables so that the desired outcomes are met. Here, the research becomes 
longitudinal because social concerns can only be effectively dealt with iu-the-long- 
run. The authors advocate the second alternative because it is prevention oriented. 
This is not to contend that some of this type of work is not going on. An example 
would be the Engleman and Becker system for teaching math, reading and language. It 
effectively solves a social problem and prevents, where ever it is used, many of the 
problems that previously occurred in these settings. 

This paper has attempted to deal with two problems confronting the applied 
researcher using a behavior analysis approach. The first was the correlated behaviors 
problem. The effectiveness with which the researcher can solve this type of problem, 
it was argued, depends on the researcher's emphasis on the examination of procedures 
and not experimental design. Next, it was argued that to maximize success in .dealing 
with these problems one must also see the possible contribution the researcher can 
add to the complex of empirical knowledge on human behavior. This latter point required 
a particular perspective related to the basic-applied research distinction. 



APPENDIX 

The following section is a review of behavior analysis designs and possible 
sources of invalidity proposed by Campbell and Stanley. We should emphasize that the 
sources of invalidity as applied to behavior analysis designs in many cases is 
inappropriate. As reviewed in the previous section the criticism of the designs is 
unwarranted because the real problem lies with the development of appropriate 
procedures. 
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Behavior Analysis Designs 

First, let us briefly review the 12 factors that Campbell and Stanley cite 
as possible sources of invalidity • The first eight refer to internal validity, and 
the last four refer to external validity • These factors are as follows: 

1. History, the specific events occurring between the first and second measure- 
ment. These events usually occur in the environment. 

2. Maturation, the processes within the subject operating as a function of the 
passing of time. These processes are biological (as opposed to environmental). 

3. Testing, the effects of taking a test upon the scores of a second testing. 

4. Instrumentation, any changes due to observer effects or changes in the 
calibration of the measuring instrument. 

5. Statistical regression, operates where groups have been selected on the 
basis of extreme scores. 

6. Selection, any biases resulting from the differential selection of subjects 

for comparison groups. 

7. Mortality, any differential loss of subjects from comparison groups. 

8. Selection — maturation, interaction, etc., any confounding effects with the 
experimental variable. 

9. Interaction effect of testing and treatment, where a pretest might increase 
or decrease the subject's sensitivity or responsiveness to the experimental variable. 

10. Interaction effect of selection and treatment, where selection may interact 
with the treatment to produce a confounding variable. 

11, Reactive arrangements, deals with generalizability in non-experimental 
settings. It also deals with the presence or absence of observers in experimental 
settings. 

12. Multiple-treatment interference, what are the effects of prior treatments? 
These 12 factors will be used to analyze the basic designs that are employed in 
behavior analysis studies. 
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There are a number of research designs that are used by experimenters to 
analyze behavior in a naturalistic setting. They are (1) A-B reversal, (2) A-B-A-B 
reversal, and (3) ^Multiple Baselines. Let us now look at each of these designs with 
regard to threats to internal and external validity. 

1. A-B Reversal Design (Figure 1) 

In this design, a behavior is recorded for an operant level (baseline or A) 
and then some contingency or change is instated (B) . The way in which the behavior 
changes from condition A to condition B for an individual subject determines the 
strength of the causal inference made by the researcher. 

The strength of this inference is open to possible attacks to its internal 
validity. The first confounding factor is subject history, because other change- 
producing events in the environment may have occurred in addition to the experimenter's 
manipulations. A second possible confounding could be due to maturation. or change- 
producing biological events. A third confounding may be due to an interaction of 
maturation and selection. The other Campbell and Stanley sources of threats to 
^ internal validity are either controlled for or are not applicable. Testing is 

controlled for through the use of multiple baselines. Behavior is recorded contin- 
uously as opposed to a pretest, post-test recording. Instrumentation is controlled 
for also. Instrumentation is not a design problem per se but a procedural problem 
for experimental analysis designs, and is usually taken care of through the use of 
• individual subjects. The factor of selection is irrelevant because of the use of 
an individual subject and not groups. There is no comparison group, and therefore, 
there is no need to worry about whether or not the experimental groups are the same. 
Our concern is only with the individual subject. Finally, mortality is not applicable 
because of the emphasis on the individual subject. If that subject drops out, there 
is no experiment to be run! 

With regard to external validity, the A-B reversal design is very weak. It is 
not generalizable beyond the individual subject in a specific setting. One final j j 
O „ problem with this design deals with the question of what would have happened if the 
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investigator had not introduced a treatment? In other words, the investigator would 
want to know if the subject's baseline behavior would have increased, decreased, or 
remained the same if a treatment had not been instated. The answer to this problem 
would control the threats of history, maturation, and selection by maturation interaction. 

2. A-B-A-B Reversal Design (Figure II and III) 

This design is a four-stage experimental design. It attempts to establish 
sufficient controls in order to make strong inferences. This is how the design 
works: (1) take a baseline—first A, (2) instate a contingency— first B, (3) reverse 
back to the baseline condition — second A, and (4) reverse back to the contingency — 
second B. This design will allow one to make statements about the sufficiency of the 
conditions to cause the change that is observed. Wolf and Risley (1971) state that 
if the behavior reverses back to something approximating the baseline level when 
the treatment is withdrawn, one can make a reasonalbe estimate about what the natural 
course of the uninterrupted baseline behavior would have been. In this design, the 
behavioral changes (expressed in terms of ongoing variability) are or are not shown 
to be a function of the repeated presentation (the B's) and removal (the A's) of the 
experimental condition. 

With regard to internal validity, the A-BeA-B reversal design controls for the 
factors of history, maturation, and the interaction of selection by maturation. 
These controls are accomplished through the use of reversal. With the reversals, 
there is a measure across time which allows us to see what happens when the contingency 
is removed. Given appropriate procedures, the other threats to internal validity 
are controlled for or are not applicable (see A-B reversal design). 

In reference to external validity, one cannot generalize beyond the subject in 
a specific situation. The reason for this, is because of the lack of random sampling. 
One way around this problem is to employ a Comfield-Tukey logic argument. If the 
experimenter adequately describes the subject, baseline, and situation, he may then 
generalize his findings to similar subjects and situations. One final problem with 
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this design is the inability to examine irreversible behavior, be it functional (a 
true behavioral process) or practical (for some reason you just do not want to reverse 
the behavior). This design can also be elaborated if the treatment contains a number 
of components. In this way one can determine the amount of variability controlled by 
the individual components or combination of components (Figure III). How the components 
are introduced is determined by the questions the research wants to answer. 
The multiple baseline designs were developed to overcome this last problem. 

3. Multiple baselines are experimental designs in which data are collected: 
(a) across behaviors, (b) across individuals, or (c) across situations - 
a- Multiple Baselines Across Behaviors (Figure IV) 
In this design, two or more behaviors are observed ior a single subject (multi*? 
pie baselines). The treatment condition is then introduced for one of the behaviors 
but not for the second behavior. In this way, the second behavior acts as a control 
for.the first behavior. The treatment condition is later introduced for the second 
behavior. In this fashion, the greater the number of baselines, the greater our 
confidence can be in asserting that the effect was in fact due to our treatment 
(internal validity) and not to some confounding variable. The chance of confounding 
the interpretations of this design is slight because of the differential and systematic 
introduction of experimental conditions. This design allows a cause and effect 
relationship to be made about behaviors. The experimenter, by using two or more 
behaviors in his design, can now make stronger inferential statements about behavior 
than he could in the A-B or A~B-A-B reversal designs. 

Given proper procedures, the internal validity of multiple baselines is very 
strong, (reviewed in first part of this paper) 

The external validity of this design is stronger than the reversal designs 
because the experimenter can now generalize his findings to more than one behavior. 
Uis inferences are stronger because the experimental condition is applicable to 
more than one behavior. 

13 
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b. Multiple Baseline Across Individuals (Figure V) 

In this design, baseline data are collected for the same behavior of different 
subjects in the same situation. The treatment condition is introduced for one of 
the subjects; but it is not introduced for the other subjects, the purpose being 
that the behavior of a subject will not change until he is given the treatment. 
The treatment condition is later introduced to the other subjects one at a time. 
When one subject is given the treatment condition, the other subjects are very much 
like a control group in a group comparison study. 

Threats to internal validity, again given proper procedures, may easily be avoided 
With regard to external valiflit>, this design extends the subject (or individual) 
generalizations beyond the N = 1 findings. Since we have more than one subject, 
we can generalize our findings to the other subjects as well. If a population of 
subjects was defined and a ranodm sample drawn, we could then generalize our findings 
to our population. The same holds for multiple baselines across behaviors and across 
situations. This design uses both intra- and inter-subject replication to give 
^;reater inferenctial strength. 

c. Multiple Baselines Across Situations (Figure VI) 

In this design, baseline data are collected for the same behavior of one subject 
in different situations. The treatment condition is introduced in one of the situa- 
tions but not in the other situations. The treatment condition is later introduced 
in the other situations. The main interest here is to see how an individual behavior 
changes in different situations. 

Given proper procedures, threats to internal validity and external valid-rty are 
minimal. The more situations in which the experimental conditions are instated, the 
greater the situation generalization. With this design, the experimenter could define 
a population of situations and randomly sample from this population for use in the 
study. The i..ferences about the treatment condition could then be applied to the 
population. 
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d. Complex Multiple baseline across behaviors, subjects, and situations 
(Figure VII). 

With this design we can achieve behavior, subject, and situation generality. 
Given consistent results this design is strongest in establishing external validity. 
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Figure li A-B Reveral Designs 
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Figure lit A-B-A-B Reversal Designs 
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Figure nil A-B^A-n Reversal Design with Component Analysis 
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Figure 



IV 1 Multiple Baseline Design 
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ngure V, Multiple Ba.ellne Design for Subject Generality 
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Figure 



for Situation Generality 
VI. Multiple Baseline Design for 
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